Intel Xeon X7560: Nehalem EX Review

System Architecture

One of the biggest features (pun intended) of the Nehalem EX architecture is its ability to support up to 256 CPUs in a single motherboard. This is massively important (pun, once again intended), as only two Xeon 5600-series CPUs or four Xeon 7400-series processors can be installed in a motherboard.

Historically, this was a real problem for Intel as its earlier MP processors, the Xeon 7400-series, didn’t scale very well in performance beyond two CPUs. This was because the CPUs could not communicate directly with one another; instead they had to share data via a single slow 533MHz FSB through the Intel 7300 chipset in the motherboard.

Worse still, all four CPU sockets were forced to share a single memory controller, so all 24 cores would fight for contention of the dual-channel 667MHz DDR2 memory. This caused a horrific bottleneck, massively restricting performance (once again, pun intended). In short, the Xeon 7400-series wasn’t very good at all, and by all reports it didn’t sell very well either.

So with eight physical cores plus eight logical Hyper-Threaded cores per processor, and up to 256 CPUs in a single motherboard, Intel clearly needed to do something radical to make the Xeon 7500-series more competitive than its predecessor.

The QPI bus, first introduced with the Nehalem-architecture Core i7 desktop CPUs, provided the answer to this problem.

The QPI bus is hugely important for Intel, not only because it’s eight times faster than the 533MHz FSB used by the Xeon 7400-series, but it’s also scalable, so there can be more than one QPI link in a system.

Intel tried this out first on the Xeon 5500-/5600-series, giving each CPU two QPI links – one to talk to the other CPU and one to talk to the motherboard. For the Xeon 7500-series this model has been scaled up even further, so that each CPU has four QPI links; three to talk to other CPUs and one to talk to the motherboard.

This also means that each CPU can access each other’s cache and RAM, greatly improving performance in massively multi-threaded applications.



Now that Intel finally has a CPU and system architecture that can efficiently scale beyond two CPUs, it’s also tried to make the Xeon 7500-series more attractive to IT managers by increasing the reliability of the platform too.

Thus, the Xeon 7500-series boasts several features that, while common on IA-64 (Itanium) and RISC servers, have never been seen before in an x86 system.

These include several improvements to the CRC (cyclic redundancy check) of the QPI bus and the ability to disable individual CPU cores during POST so you can still use the system. Of the list of 20 or so other improvements that Intel is most excited about is MCA recovery.

MCA (machine check architecture) is designed to allow the system to recover from a correctable hardware error, such as a problem with a specific CPU core or DIMM, by reporting this error to the operating system or VMM (virtual machine manager). The OS or VMM can than recover without crashing the entire system.